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The  Pareto  multiple  query  image  retrieval  software  implementing  the  method  reported  in  publication  [26]  was  transitioned  to 
ARL  and  is  under  evaluation. 

Supported  student  Brandon  Oselio  did  an  internship  at  ARL  under  Lance  Kaplan  in  summer  2014. 

Alfred  Hero  visited  Army  Research  Laboratory  for  one  day  in  Feb  2013.  He  had  discussions  with  Lance  Kaplan,  Brian  Sadler, 
Nasser  Nasrabadi,  Terry  Moore  and  Tien  Pham  on  topics  relevant  to  this  project. 

Two  of  Alfred  Hero’s  PhD  students  (Brandon  Oselio  and  Ted  Tsiligaridis)  visited  ARL  as  interns  in  summer  2013. 

Alfred  Hero  has  had  several  meetings  with  ARL  researchers  Brian  Sadler,  Laurel  Sadler  and  Robert  Winkler  on  designing  a 
human  factors  experiment  at  ARL  using  the  Pareto  frontier  search  engine  (https://dl.dropboxusercontent. 
com/u/1361 1327/paretoretrievalgui.zip)  developed  under  this  project. 

Alfred  Hero  helped  Liyi  Dai  organize  a  Government  Panel  of  DoD  researchers  and  research  managers  in  Ann  Arbor  at  the  IEEE 
Statistical  Signal  Processing  Workshop  (A  Hero  was  General  Chair). 

Alfred  Hero  visited  Army  Research  Laboratory  for  one  day  in  January  2012.  He  had  discussions  with  Lance  Kaplan,  Brian 
Sadler,  Nasser  Nasrabadi,  Terry  Moore  on  topics  relevant  to  this  project. 

Two  of  Alfred  Hero’s  PhD  students  visited  ARL  as  interns  in  summer  2012. 

Alfred  Hero  met  with  ARL  researchers  Brian  Sadler  and  Terry  Moore  at  College  Park  during  the  FFT  symposium  held  in  the 
Dept  of  Mathematics  at  Univ  of  Maryland. 

Alfred  Hero  organized  an  ARO  sponsored  workshop  in  April  in  Annarbor  on  Sensor  Information  Estimation  and  Exploitation  with 
the  help  of  Army  research  and  development  center  researcher  Donald  Waagen. 

Alfred  Hero  helped  Liyi  Dai  organize  a  Government  Panel  of  DoD  researchers  and  research  managers  in  Ann  Arbor  at  the  IEEE 
Statistical  Signal  Processing  Workshop  (A  Hero  was  General  Chair). 
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Overall  objective  of  project: 

The  objective  is  to  develop  sparse  representation  and  dimensionality  reduction  methods  for  robust  data  mining 
and  indexing  of  multimodality  image  and  video  databases.  The  near  term  objective  has  been  to  establish 
feasibility  of  our  information  theoretic  dimensionality  reduction  approach  to  this  complex  problem.  The 
ultimate  objective  is  to  apply  our  techniques  to  large  image  and  video  databases,  centralized  or  possibly 
distributed,  in  order  to  reduce  search  complexity,  accelerate  query  matching  times,  improve  specificity  of  the 
query  matches,  and  incorporate  robustness  to  noise,  sampling,  view  point,  occlusion,  and  other  distortions  in  the 
database.  Viewpoint  and  pose  has  been  a  main  focus  of  the  project  over  the  past  two  years.  Experimental 
validation  has  been  and  will  continue  to  be  performed  by  a  combination  of  simulation  and  experiment  on  single 
and  multi  modal  databases. 

Our  approach: 

We  adopted  information-theoretic  measures,  distortion  invariant  representations,  sparse  approximation,  and 
dimensionality  reduction  for  our  work.  Sparsity  and  dimensionality  reduction  have  been  successfully  applied  to 
other  problem  domains  to  reduce  complexity.  We  are  demonstrating  that  indexing  of  large  multimodality  sensing 
databases  can  also  benefit  from  this  approach.  Furthermore,  since  our  proposed  methods  are  strongly  grounded 
in  information  theory,  statistical  signal  processing,  and  machine  learning,  our  algorithms  can  provide 
quantitative  theoretical  performance  guarantees  under  the  best  conditions  (matched  model  assumptions)  and  the 
worst  conditions  (mismatched  model  assumptions).  This  has  led  to  a  better  understanding  of  the  factors  that  limit 
system  performance,  as  measured  by  a  number  of  criteria  such  as:  data  mining  false  discoveries,  search 
complexity,  and  probability  of  retrieval  classification  error.  This  theoretical  component  is  a  strength  of  this  team 
and  a  key  feature  of  the  proposed  research. 

Scientific  barriers: 

For  many  Army  applications  it  is  especially  important  to  have  robust  data  mining  and  retrieval  algorithms  with 
guaranteed  high  sensitivity  and  specificity  for  complex  databases.  However,  current  algorithms  are  unable  to 
deal  effectively  with  complex  databases  arising  in  these  applications.  An  example  is  where  the  algorithm  must 
fuse  information  across  different  modalities.  For  instance  when  the  user  supplies  a  query  picture  from  one 
modality,  e.g.,  IR,  while  the  most  relevant  entries  of  the  database  may  come  from  a  different  modality,  e.g., 
Radar.  Another  example  is  mining,  searching,  and  indexing  databases  of  high  dimensional  objects  that  are  not 
properly  registered,  e.g.,  videos  that  contain  common  scenes  or  events  that  were  acquired  at  different  times,  with 
different  resolutions,  different  viewing  angles,  or  under  different  lighting  conditions.  A  third  example  is  the 
common  situation  where  there  is  significant  uncertainty  in  the  entries  due  to  noise,  insufficient  samples, 
occlusion,  or  limited  spatial  resolution.  While  we  may  not  specifically  address  all  of  these  applications  in  this 
project,  our  general  dimensionality  reduction  framework  will  supply  tools  that  can  be  applied  to  them. 


Significance: 


The  framework  for  mining  and  indexing  complex  databases  enables  new  tools  that  are  being  applied  to  improve 
database  object  retrieval  and  discovery  performance.  These  tools  are  being  used  to  address  problems  of  high 
significance  to  the  Army  such  as  multimodality  image  retrieval,  anomaly  detection  in  videos,  and  event 
correlation  in  videos. 

Cumulative  summary  of  accomplishments  over  the  period  of  the  grant  (07/01/09-06/30/14) 

We  have  demonstrated  the  capability  of  sparse  representation  and  dimensionality  reduction  to  make  significant 
improvements  in  multi-modality  computer  vision  and  information  retrieval  problems,  especially  in  spatio- 
temporal  databases.  Our  non-incremental  research  has  resulted  in  several  major  advances  in  both  theory  and 
practice.  Our  fundamental  contributions  to  the  mathematical  theory  of  multi-objective  database  indexing  and 
retrieval  have  led  to  more  effective  ways  for  humans  to  search  image  databases  when  there  arc  multiple  queries. 
This  method  has  been  applied  to  anomaly  detection  in  surveillance  videos,  published  in  the  top  conference  on 
machine  learning  (NIPS),  multi-query  image  database  search,  submitted  to  IEEE  Trans.  On  Image  Processing, 
and  social  collaborative  retrieval,  which  appeared  in  IEEE  Journal  on  Selected  Topics  of  Signal  Processing.  A 
remarkable  theoretical  limit  was  discovered  that  specifies  a  novel  way  to  perform  indexing:  solve  a  novel 
Hamilton  Jacobi  partial  differential  equation  instead  of  performing  a  combinatorial  optimization  for  searching 
for  best  query  matches.  This  theory  was  published  in  a  top  math  journal  (SIAM  Math  Analysis)  and  its  indexing 
implementation  was  submitted  to  another  top  math  journal  (SIAM  Numerical  Analysis).  Our  development  of 
new  sparse  non-parametric  multimodality  similarity  measures  (entropy,  directed  information,  and  measure- 
transformed  CCA)  has  been  published  in  the  top  machine  learning  conference  (NIPS)  and  in  the  top  journals  on 
multi-media  (IEEE  Trans.  On  Multimedia)  and  signal  processing  (IEEE  Trans.  On  Signal  Processing).  Our 
research  on  visualizing  and  modeling  dynamic  graphs  has  been  published  in  two  papers  appealing  in  the  top 
journal  on  knowledge  discovery  and  data  mining  (Journal  of  DMKD).  Soft  information,  such  as  relational 
annotations  between  objects  in  database,  is  often  instantiated  by  graphs.  Another  major  line  of  work  supported 
by  this  grant  has  focused  on  object  detection,  pose  estimation  and  classification  using  machine  learning  tools 
including  a  novel  formulation  of  the  branch  and  bound  algorithm.  This  has  led  to  publications  to  the 
International  Conference  in  Computer  vision  (ICCV)  in  2011,  Computer  Vision  and  Pattern  Recognition 
(CVPR)  conference  in  2012,  IASTATS  2012  and  CVPR  2013.  More  recently,  our  research  on  hierarchical 
methods  for  large  scale  object  classification  was  published  at  ICCV  in  December  2013.  Finally,  our  framework 
for  joint  object  detection,  pose  estimation  and  fine-grain  categorization  has  been  submitted  to  NIPS  in  2014  and 
is  currently  under  review. 


Specific  accomplishments  over  the  period  (08/1/13-06/30/14): 

This  past  year  has  seen  substantial  progress  that  caps  this  5  year  project.  Specifically,  Hero  reports  progress  on 
modeling  and  annotating  databases  of  dynamic  graphs,  relevant  to  understanding  complex  time  varying 
relational  objects,  e.g.,  interactions  between  groups  of  people  or  correlations  between  different  types  of  observed 
events.  Hero  also  reports  progress  on  multi-objective  indexing  and  retrieval.  These  are  in  addition  to  new 
publications  in  the  areas  of  multimodality  databases  [27,28]  and  applications  [29,  30]  of  non-linear  similarity 
measures  developed  under  this  grant.  Savarese  reports  results  on  localization  of  3D  Objects  from  a  single  D- 
RGB  image,  large  scale  object  classification  and  joint  pose  estimation  and  object  fine-grained  categorization. 
Several  publications  have  been  submitted  or  have  appeared  [l]-[20]  on  work  funded  under  this  grant.  The  work 
reported  in  publications  [1]-[21]  was  described  in  previous  progress  reports.  Here  we  describe  progress  reported 
in  publications  [22] -[32]  submitted  in  the  past  year. 

Progress  1:  Mining  databases  of  dynamic  graphs  [21,22,23] 

Data  that  is  relational  is  often  represented  as  a  graph  with  nodes  and  edges.  For  example,  this  could  be  a  relational 
database  of  various  groups  of  people,  places,  and  activities.  Each  group's  pairwise  interactions  are  represented  by  a 
graph  whose  nodes  denote  people  and  whose  edges  represent  proximity  or  activity  between  the  people,  e.g., 
closeness  in  space  or  high  frequency  of  activity-specific  interaction.  These  relations  generally  evolve  over  time 
and  the  changes  in  graph  connectivity  and  shape  contain  important  cues  on  evolution  of  the  community  structure  of 


the  people  in  the  database.  We  have  focused  on  three  components  of  the  problem  of  indexing  and  retrieval  of  such 
graph-valued  objects  from  databases:  1)  mathematical  methods  for  visualization  of  dynamic  graphs,  important  for 
human-interpretation  of  objects  in  the  database;  2)  scalable  predictive  modeling  of  dynamic  graphs,  important  for 
detecting  anomalous  graph  behaviors  over  time;  and  3)  node  and  edge  centrality  measures,  important  for 
annotation  of  databases  of  graphs  according  to  graph  topology.  Last  year  we  published  our  solution  of  the 
dynamic  graph  visualization  problem  in  the  leading  journal  on  data  mining  [12].  This  year  we  have  published  our 
solution  to  scalable  predictive  modeling  using  dynamic  stochastic  bock  models  [21],  published  in  the  same  journal 
as  [12].  We  have  also  published  two  papers  [22,23]  that  introduce  a  new  measure  of  graph  centrality,  called  local 
Fiedler  centrality,  that  can  be  used  to  better  index  graph-valued  objects  with  respect  to  community  structure 
(modularity)  of  their  topologies. 

Progress  2:  Multi-objective  retrieval  and  indexing  [24,25,26] 

Over  the  past  four  years  we  have  been  constructing  fundamental  theory  and  practice  that  underpins  an  alternative 
indexing  and  retrieval  method,  which  we  call  multi-objective  retrieval  and  indexing.  The  theory  has  been 
published  in  a  leading  journal  on  mathematics  analysis,  SIAM  Journal  of  Mathematical  Analysis  [17],  and  a 
numerical  scheme  that  transitions  the  theory  in  [17]  to  practice  was  submitted  last  year  to  another  leading 
mathematics  journal,  SIAM  Journal  of  Numerical  Analysis  [24].  These  fundamental  mathematical  results  are 
remarkable  since  they  specify  a  continuum  limit  (a  partial  differential  equation)  for  the  combinatorial  Pareto 
ranking  problem  that  is  the  crux  of  multi-objective  retrieval  and  indexing.  This  year  we  published  a  synopsis  of 
the  mathematical  result  for  engineers,  which  appeared  in  the  proceedings  of  one  pf  the  premier  (by-invitation- 
only)  conferences  [25]  in  the  applications  of  information  theory.  In  the  SIAM  paper  [24]  we  demonstrated  that 
the  pde  continuum  approximation  leads  to  significant  acceleration  of  multi-objective  video  anomaly  detection, 
specifically  the  method  introduced  in  [14],  We  have  submitted  another  indexing  and  retrieval  application  of  this 
theory  to  the  premier  journal  on  image  processing  [26].  This  application,  called  multiple  query  image  retrieval, 
allows  a  user  to  peruse  a  database  using  Pareto  partial  ordering  instead  of  the  standard  linear  ordering  used  by 
today’s  image  search  engines.  The  software  for  multiple  query  image  retrieval  has  been  transitioned  to  ARL 
and  is  currently  under  evaluation. 


Progress  4:  Accurate  Localization  of  3D  Objects  from  RGB-D  Data  using  Segmentation  Hypotheses  [17] 

In  this  work  we  focus  on  the  problem  of  finding  objects  in  3D  from  RGB-D  images.  We  propose  a  novel 
framework  that  explores  the  compatibility  between  segmentation  hypotheses  of  the  object  in  the  image  and  the 
corresponding  3D  map.  Our  framework  allows  discovering  the  optimal  location  of  the  object  in  3D  using  a 
generalization  of  the  structural  latent  SVM  formulation  in  3D  as  well  as  the  definition  of  a  new  loss  function 
defined  over  3D  space  in  training.  We  evaluate  our  method  using  two  existing  RGB-D  datasets.  Extensive 
quantitative  and  qualitative  experimental  results  show  that  our  proposed  approach  outperforms  state-of-the-art  as 
well  as  a  number  of  baseline  methods  for  both  3D  and  2D  object  recognition  tasks.  This  work  was  presented  at 
the  IEEE  conference  in  Computer  Vision  and  Pattern  Recognition  (CVPR,  June  2013)  [17] 


Progress  5:  An  Efficient  and  Accurate  Classifier  for  Image  Hierarchies  [31].  Many  methods  have  been 
proposed  to  solve  the  image  classification  problem  for  a  large  number  of  categories.  Among  them,  methods 
based  on  tree-based  representations  achieve  good  trade-off  between  accuracy  and  test  time  efficiency.  While 
focusing  on  learning  a  tree-shaped  hierarchy  and  the  corresponding  set  of  classifiers,  most  of  them  use  a  greedy 
prediction  algorithm  for  test  time  efficiency.  We  argue  that  the  dramatic  decrease  in  accuracy  at  high  efficiency 
is  caused  by  the  specific  design  choice  of  the  learning  and  greedy  prediction  algorithms.  In  this  work,  we 
propose  a  classifier  which  achieves  a  better  trade-off  between  efficiency  and  accuracy  with  a  given  tree-shaped 
hierarchy.  First,  we  convert  the  classification  problem  as  finding  the  best  path  in  the  hierarchy,  and  a  novel 
branch-and-bound-like  algorithm  is  introduced  to  efficiently  search  for  the  best  path.  Second,  we  jointly  train  the 
classifiers  using  a  novel  Structured  SVM  (SSVM)  formulation  with  additional  bound  constraints.  As  a  result,  our 
method  achieves  improvement  in  accuracy  at  high  efficiency  compared  to  state-of-the-art  greedy  “tree-based” 
methods  dataset,  respectively.  This  work  was  presented  at  the  ICCV  conference  in  December  2013  [31] 


Progress  6:  A  Unified  Framework  for  Object  Detection,  3D  Pose  Estimation,  and  Sub-category  Recognition 
[32],  Object  detection,  3D  pose  estimation,  and  sub-category  recognition  are  usually  highly  correlated  tasks. 
However,  these  tasks  are  typically  addressed  independently  from  each  other  because  of  the  huge  space  of 
parameters  when  modeled  jointly.  To  jointly  model  all  of  these  tasks,  we  propose  a  coarse-to-fine  hierarchical 
representation,  where  each  level  of  the  hierarchy  represents  objects  at  a  different  level  of  granularity.  The 
hierarchical  representation  prevents  performance  loss,  which  is  often  caused  by  the  increase  in  the  number  of 
parameters  (as  we  consider  more  tasks  to  model),  and  the  joint  modeling  enables  resolving  ambiguities  that  exist 
in  independent  modeling  of  these  tasks.  We  augment  PASCAL3D  -  a  state-of-the-art  large  scale  3D  object 
dataset  -  with  annotations  for  these  tasks  and  show  that  our  hierarchical  model  is  effective  in  joint  modeling  of 
object  detection,  3D  pose  estimation,  and  sub-category  recognition.  This  work  has  been  submitted  to  NIPS  2014 
[31]. 
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Published  or  submitted  in  current  report  period  (8/1/13-6/30/14): 

[21]  K.S.  Xu,  M.  Kliger,  AO  Hero,  "Adaptive  evolutionary  clustering,"  J.  Data  Mining  and  Knowledge 
Discovery,  Volume  29,  Issue  2,  pp.  304-336,  2014.  Available  as  arXiv:  1104. 1990 

[22]  P.-Y.  Chen  and  A.O.  Hero,  "Local  Fiedler  Vector  Centrality  for  Detection  of  Deep  and 
Overlapping  Communities  in  Networks,"  Proc.  of  IEEE  Conf.  on  Acoustics,  Speech  and  Signal 
Processing  (ICASSP)  2014.  Florence  Italy,  May  2014.  ARO-databases. 
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[24]  J.  Calder,  S.  Esedoglu,  A.  O.  Hero,  "A  PDE-based  approach  to  non-dominated  sorting,"  available 
as  arxiv:  1320. 2498(Oct  2013).  Submitted  to  SIAM  Numerical  Analysis. 

[25]  J.  Calder,  S.  Esedoglu  and  A.O.  Hero,  "A  continuum  limit  for  non-dominated  sorting,”  Conference 
on  Information  Theory  and  its  Applications  (ITA),  San  Diego,  Feb  2014. 

[26]  K.J.  Hsiao,  J.  Calder  and  A.O.  Hero,  "Pareto-depth  for  multiple-query  image  retrieval,"  Submitted. 
Available  as  arxiv:  1402.5176  (.html) .  2014. 


[27]  K.-J.  Hsiao,  J.  Kulezsa  and  A.O.  Hero,  "Social  Collaborative  Retrieval,”  IEEE  Journal  on  Selected 
Topics  in  Signal  Processing,  Vol.  8,  No.  4,  pp.  680  -  689,  2014.  Available  as  arxiv:  1404.2342  (.html) 


[28]  X.  Chen,  A.  Hero,  S.  Saverese,  "Shrinkage  Optimized  Directed  Information  using  Pictorial 
Structures  for  Action  Recognition,"  Technical  Report,  2014.  Available  as  arxiv:  1404.33 12  (.html)  . 

[29]  X.  Chen,  S.  Syed,  and  AO  Hero,  "EEG  Spatial  Decoding  and  Classification  with  Logit  Shrinkage 
Regularized  Directed  Information  Assessment  (L-SODA)."  Technical  Report,  2014.  Available  as 
arXiv:  1404.0404  (.html)  . 

[30]  K.  Todros  and  A.O.  Hero,  "Robust  Measure  Transformed  MUSIC  for  DOA  Estimation,"  Proc.  of 
IEEE  Conf.  on  Acoustics,  Speech  and  Signal  Processing  (ICASSP)  2014. 

[31]  M.  Sun,  W.  Huang,  and  S.  Savarese,  Find  the  Best  Path:  an  Efficient  and  Accurate  Classifier  for 
Image  Hierarchies ,  Proceedings  of  the  International  Conference  in  Computer  Vision  (ICCV)  2013. 

[32]  R.  Mottaghi,  S.  Savarese,  An  Unified  Framework  for  Object  Detection,  3D  Pose  Estimation,  and 
Sub-category  Recognition,  NIPS  (submitted)  2014 


DATA  REQUIRED 

(1)  Submissions  or  publications  under  ARO  sponsorship  during  this  reporting  period.  List  the  title  of  each  and 
give  the  total  number  for  each  of  the  following  categories: 

(a)  Papers  published  in  peer-reviewed  journals:  3 

K.S.  Xu,  M.  Kliger,  AO  Hero,  "Adaptive  evolutionary  clustering,"  J.  Data  Mining  and  Knowledge  Discovery, 
Volume  29,  Issue  2,  pp.  304-336,  2014.  Available  as  arXiv:  1104.1990 

J.  Calder,  S.  Esedoglu,  A.  O.  Hero,  "A  PDE-based  approach  to  non-dominated  sorting,”  to  appear  in 
SIAM  Numerical  Analysis.  Available  as  arxiv:  1 320. 2498(Oct  2013). 

K. -J.  Hsiao,  J.  Kulezsa  and  A.O.  Hero,  "Social  Collaborative  Retrieval,”  IEEE  Journal  on  Selected 
Topics  in  Signal  Processing,  Vol.  8,  No.  4,  pp.  680  -  689,  2014.  Available  as  arxiv:  1404.2342  (.html) 

(b)  Papers  published  in  non-peer-reviewed  journals:  0 

(c)  Presentations 

i.  Presentations  at  meetings,  but  not  published  in  Conference  Proceedings  4 

Alfred  Hero,  “Small  sample  community  detection  in  massive  data  sets,”  Plenary  talk,  IEEE  CAMSAP 
Workshop,  St  Martins.  Dec  2013. 

Alfred  Hero,  “Spatio-temporal  graphical  models  for  high  dimensional  network  data,”  Keynote  talk,  Network 
Theory  Symposium,  IEEE  GlobalSIP  Conference,  Austin  TX.,  Dec  2013. 

Alfred  Hero,  “Sparsity  regularized  image  reconstruction,”  Quantitative  Non-Destructive  Evaluation  (QNDE), 
Keynote  talk,  Boise  ID,  July  2014. 

Alfred  Hero,  “Correlation  mining  in  large  networks  with  limited  samples,”  Keynote  talk,  IEEE  International 
Telecommunications  Symposium,  Sao  Paolo  Brazil.  Aug.  2014. 


Silvio  Savarese,  “Perceiving  the  3D  world”,  invited  talk  at  the  Large  Scale  Visual  Commerce  workshop,  in 
conjunction  of  ICCV  2013,  Australia,  December  2013 


ii.  Non-Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts):  1 

J.  Calder,  S.  Esedoglu  and  A.O.  Hero,  "A  continuum  limit  for  non-dominated  sorting,”  Conference  on 
Information  Theory  and  its  Applications  (ITA),  San  Diego,  Feb  2014. 

iii.  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts):  5 

P.-Y.  Chen  and  A.O.  Hero,  "Local  Fiedler  Vector  Centrality  for  Detection  of  Deep  and  Overlapping 
Communities  in  Networks,"  Proc.  of  IEEE  Conf.  on  Acoustics,  Speech  and  Signal  Processing 
(ICASSP)  2014.  Florence  Italy,  May  2014.  ARO-databases. 

P.-Y.  Chen  and  A.O.  Hero,  "Node  Removal  Vulnerability  of  the  Largest  Component  of  a  Network," 
Proceedings  of  IEEE  GlobalSIP  Austin  TX  Dec  2013. 

K.  Todros  and  A.O.  Hero,  "Robust  Measure  Transformed  MUSIC  for  DOA  Estimation,"  Proc.  of  IEEE 
Conf.  on  Acoustics,  Speech  and  Signal  Processing  (ICASSP)  2014. 

M.  Sun,  W.  Huang,  and  S.  Savarese,  Find  the  Best  Path:  an  Efficient  and  Accurate  Classifier  for  Image 
Hierarchies ,  Proceedings  of  the  International  Conference  in  Computer  Vision  (ICCV)  2013. 

R.  Mottaghi,  S.  Savarese,  An  Unified  Framework  for  Object  Detection,  3D  Pose  Estimation,  and  Sub- 
category  Recognition,  NIPS  (submitted)  2014 

(d)  Manuscripts  1 

J.  Calder,  S.  Esedoglu,  A.  O.  Hero,  "A  PDE-based  approach  to  non-dominated  sorting,"  available  as 
arxiv:1320.2498(Oct  2013).  Submitted  to  SIAM  Numerical  Analysis. 

(e)  Books 

(f)  Honors  and  Awards 

Lifetime  career  award:  IEEE  Signal  Processing  Society  Technical  Achievement  Award,  given  at  the  2014  Inti. 
Conf  on  Acoustic,  Speech,  and  Signal  Processing,  Florence  Italy.  May  2014. 

Best  paper  award:  IEEE  CAMSAP  2013  Best  Student  Paper  Competition,  awarded  2nd  place  for  a  paper 
coauthored  with  my  former  student  Zhaoshi  Meng  and  former  post-docs  Dennis  Wei  and  Ami  Wiesel  entitled 
“Marginal  Likelihoods  for  Distributed  Estimation  of  Graphical  Model  Parameters,”  IEEE  Computational 
Advances  in  Multi-Sensor  Adaptive  Processing  workshop,  St  Martins.  Dec  2013. 

Best  paper  award:  IEEE  ICIP  2013  Best  Paper  Award,  for  a  paper  co-authored  with  former  student  Paul  Shearer 
and  colleague  Anna  Gilbert  entitled  “Correcting  Camera  Shake  by  Incremental  Sparse  Edge  Approximation,”  at 
the  2013  IEEE  Inti.  Conf.  on  Image  Processing,  Melbourne  Australia.  Sept.  2013. 

Plenary  talk:  “Small  sample  community  detection  in  massive  data  sets,”  IEEE  CAMSAP  Workshop,  St  Martins. 
Dec  2013. 

Keynote:  “Resource  constrained  adaptive  sensing,”  New  Sensing  and  Statistical  Inference  Methods  Symposium, 
IEEE  GlobalSIP  Conference,  Austin  TX.  Dec  2013. 

Keynote:  “Spatio-temporal  graphical  models  for  high  dimensional  network  data,”  Network  Theory  Symposium, 
IEEE  GlobalSIP  Conference,  Austin  TX.,  Dec  2013. 


Keynote:  “Sparsity  regularized  image  reconstruction,”  Quantitative  Non-Destructive  Evaluation  (QNDE),  Boise 
ID,  My  2014. 

Keynote:  “Correlation  mining  in  large  networks  with  limited  samples,”  IEEE  International  Telecommunications 
Symposium,  Sao  Paolo  Brazil.  Aug.  2014. 

Plenary:  “Correlation  mining  for  imaging  and  multidimensional  signal  processing,”  IEEE  Inti  Conference  on 
Image  Processing,  Paris  France.  Oct.  2014 

Alfred  Hero  was  invited  to  give  6  plenary  lectures  including  the  following: 

1.  “Small  sample  community  detection  in  massive  data  sets,”  IEEE  CAMSAP  Workshop,  Dec 
2013. 

2.  “Resource  constrained  adaptive  sensing,”  New  Sensing  and  Statistical  Inference  Methods 
Symposium,  IEEE  GlobalSIP  Conference,  Dec  2013. 

3.  “Spatio-temporal  graphical  models  for  high  dimensional  network  data,”  Network  Theory 
Symposium,  IEEE  GlobalSIP  Conference,  Dec  2013. 

4.  “Information-driven  multimodality  fusion,"  AFRL/UES  Workshop  on  Data  Fusion  for  the 
Detection  of  Rare  and  Anomalous  Events,  Dec  2012." 

5.  “Graphical  modeling  for  high  dimensional  data  analysis,"  (Keynote  1)"  Solar  Information 
Processing  Workshop  (SIP)  VI,  MSU  Bozeman,  Aug.  14  2012. 

6.  “Learning  with  entropic  graphs,"  (Keynote  Solar  Information  Processing  Workshop  (SIP)  VI, 
MSU  Bozeman,  Aug.  15  2012. 

Alfred  Hero  won  a  Notable  Paper  award  for  his  work  on  graphical  models  from  AISTATS  2013  for  the 
paper:  Z.  Meng,  D.  Wei,  A.  Wiesel,  A.  Hero,  "Distributed  Learning  of  Gaussian  Graphical  Models  via 
Marginal  Likelihoods,"  AISTATS  Scottsdale  AZ.  April  2013. 

Alfred  Hero  was  awarded  the  University  of  Michigan  Rackham  Distinguished  Faculty  Achievement  Award  in 
2011. 

(g)  Title  of  Patents  Disclosed  during  the  reporting  period 

(h)  Patents  Awarded  during  the  reporting  period 

(2)  Student/Supported  Personnel  Metrics  for  this  Reporting  Period  (name,  %  supported,  %Full  Time  Equivalent 
(FTE)  support  provided  by  this  agreement,  and  total  for  each  category): 

(a)  Graduate  Students 

Liang  Mei,  50%  FTE,  EE:  systems,  PhD  candidate 
Min  Sun,  50%  FTE,  EE:  systems,  PhD  candidate 
Yu  Xiang,  50%  FTE,  EE:  systems,  PhD  candidate 
Dae  Yon  Jung,  50%  FTE,  EE:  systems,  PhD  candidate 
Ko-Jen  Hsiao,  50%  FTE,  EE:  system  PhD  candidate 
Zhaoshi  Meng:  50%  FTE,  EE:  systems,  PhD  candidate 
Yu-Wei  Chao:  50%  FTE,  EE:  system  PhD  candidate 
Jinyoung  Hwang:  50%  FTE,  EE:  systems,  PhD  candidate 
Sung  Jin  Hwang:  50%  FTE,  EE:  systems,  PhD  candidate 
Anush  Mohan:  50%  FTE,  EE:  systems,  PhD  candidate 

(b)  Post  Doctorates 

Sung  Jin  Hwang:  100%  FTE 
Xu  Chen:  50%  FTE 


John  Alexander  Kulesza:  50%  FTE 
Greg  Newstadt:  100%  FTE 
Koby  Todros;  100%  FTE 

(c)  Faculty 
Alfred  Hero 
Silvio  Savarese 

(d)  Undergraduate  Students 

(e)  Graduating  Undergraduate  Metrics  (funded  by  this  agreement  and  graduating  during  this  reporting  period): 

i.  Number  who  graduated  during  this  period 

ii.  Number  who  graduated  during  this  period  with  a  degree  in  science,  mathematics,  engineering,  or  technology 
fields 

iii.  Number  who  graduated  during  this  period  and  will  continue  to  pursue  a  graduate  or  Ph.D.  degree  in  science, 
mathematics,  engineering,  or  technology  fields 

iv.  Number  who  achieved  a  3.5  GPA  to  4.0  (4.0  max  scale) 

v.  Number  funded  by  a  DoD  funded  Center  of  Excellence  grant  for  Education,  Research  and  Engineering  0 

vi.  Number  who  intend  to  work  for  the  Department  of  Defense 

vii.  Number  who  will  receive  scholarships  or  fellowships  for  further  studies  in  science,  mathematics,  engineering 
or  technology  fields 

(f)  Masters  Degrees  Awarded  (Name  of  each.  Total  #) 

(g)  Ph.D.s  Awarded  (Name  of  each.  Total  #)  3 
Sung  Jin  Hwang,  EE:  systems,  PhD  August  2012 
Min  Sun,  EE:  systems,  PhD  May  2013 
Ko-Jen  (Mark)  Hsiao,  EE:  systems,  PhD  June  2014. 

(h)  Other  Research  staff  (Name  of  each,  FTE  %  Supported  for  each.  Total  %  Supported) 

Yue  Hou:  (Temp)  10%  FTE 

Akshay  Kumar  Jain  Pokarma:  10%  FTE 
TaoranYan:  (Temp)  5%  FTE 
Anuj  Chandawalla:  (Temp)  5%  FTE 
Chang  Wang:  (Temp)  5%  FTE 


(3)  “Technology  transfer”  (any  specific  interactions  or  developments  which  would  constitute  technology  transfer 
of  the  research  results).  Examples  include  patents,  initiation  of  a  start-up  company  based  on  research  results, 
interactions  with  industry /Army  R&D  Laboratories  or  transfer  of  information  which  might  impact  the 
development  of  products. 

Alfred  Hero  visited  ARL  in  June  2014  where  he  met  with  ARL  researchers  to  discuss,  among  other  things, 
research  under  this  grant.  ARL  researchers  with  whom  A1  Hero  interacted  included:  Lance  Kaplan,  Brian 
Sadler,  and  Nasser  Nasrabadi. 

The  Pareto  multiple  query  image  retrieval  software  implementing  the  method  reported  in  publication  [26]  was 
transitioned  to  ARL  and  is  under  evaluation. 

Supported  student  Brandon  Oselio  did  an  internship  at  ARL  under  Lance  Kaplan  in  summer  2014. 


Alfred  Hero  visited  Army  Research  Laboratory  for  one  day  in  Feb  2013.  He  had  discussions  with 
Lance  Kaplan,  Brian  Sadler,  Nasser  Nasrabadi,  Terry  Moore  and  Tien  Pham  on  topics  relevant  to  this 
project. 


Two  of  Alfred  Hero’s  PhD  students  (Brandon  Oselio  and  Ted  Tsiligaridis)  visited  ARL  as  interns  in 
summer  2013. 

Alfred  Hero  has  had  several  meetings  with  ARL  researchers  Brian  Sadler,  Laurel  Sadler  and  Robert 
Winkler  on  designing  a  human  factors  experiment  at  ARL  using  the  Pareto  frontier  search  engine 
(https://dl.dropboxusercontent.eom/u/13611327/paretoretrievalgui.zip)  developed  under  this  project. 

Alfred  Hero  helped  Liyi  Dai  organize  a  Government  Panel  of  DoD  researchers  and  research  managers 
in  Ann  Arbor  at  the  IEEE  Statistical  Signal  Processing  Workshop  (A  Hero  was  General  Chair). 

Alfred  Hero  visited  Army  Research  Laboratory  for  one  day  in  January  2012.  He  had  discussions  with 
Lance  Kaplan,  Brian  Sadler,  Nasser  Nasrabadi,  Terry  Moore  on  topics  relevant  to  this  project. 

Two  of  Alfred  Hero’s  PhD  students  visited  ARL  as  interns  in  summer  2012. 

Alfred  Hero  met  with  ARL  researchers  Brian  Sadler  and  Terry  Moore  at  College  Park  during  the  FFT 
symposium  held  in  the  Dept  of  Mathematics  at  Univ  of  Maryland. 

Alfred  Hero  organized  an  ARO  sponsored  workshop  in  April  in  Annarbor  on  Sensor  Information 
Estimation  and  Exploitation  with  the  help  of  Army  research  and  development  center  researcher  Donald 
Waagen. 

Alfred  Hero  helped  Liyi  Dai  organize  a  Government  Panel  of  DoD  researchers  and  research  managers 
in  Ann  Arbor  at  the  IEEE  Statistical  Signal  Processing  Workshop  (A  Hero  was  General  Chair). 


